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ABSTRACT 



This paper presents the results of qualitative analyses that 
were designed to assess the validity of teachers' responses to items that 
measure topic coverage in mathematics and language arts. Using data from 12 
"think- aloud" interviews and drawing on psychological theory about how 
respondents answer survey items, this research explored a number of ways that 
qualitative techniques can be used to diagnose and improve survey instruments 
in educational research. Twelve elementary school teachers in a midwestern 
U.S. city participated. Teachers were asked to report the mathematics and 
language topics they taught in a single day (unprompted report) and to chose 
the topics they had taught from a topic list. The contextual information 
teachers used in reporting topic coverage and teachers' understanding of 
reading topics were studied through the think-aloud interviews. Results 
suggest that teachers rely on a rich set of contextual information when 
recalling what they covered. Presenting teachers with the stimulus of a topic 
list triggered thoughts of content, whereas the open-ended nature of the 
unprompted report triggered the recall of other types of information. Results 
also show that teachers' under standing of the topics in topic lists cannot be 
taken for granted. Appendixes contain the reading and mathematics topic lists 
and an analysis of each teacher's understanding of two reading topics. (SLD) 
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Overview 



This paper presents the results of qualitative analyses that were designed to 
assess the validity of teachers’ responses to items that measure topic coverage in math 
and language arts. Using data from 12 “thinkaloud” interviews, and drawing upon 
psychological theory on how respondents respond to survey items, this research explores 
a number of ways that qualitative techniques can be used to diagnose and improve survey 
instruments in educational research. 

Theoretical perspectives 

Teachers’ reports of academic content coverage are considered by many to be a 
fundamental educational process variable that is indicative of student opportunity to learn 
(OTL). Given the widespread use of OTL measures as educational process indicators, 
and more recently as indicators of whether standards are being met, understanding the 
validity of such measures has important implications for policy and practice. Recent 
attempts to validate teachers’ reports of content coverage suggest that on average, 
teachers can recall topics they taught with about 75 percent accuracy (Porter et al, 1993; 
Burstein et al, 1995). Researchers in the latter study found that recall accuracy varied 
considerably by topic and believed that teachers’ understanding of topic labels (or lack 
thereof) was a key source of this variation. Beyond general measures of response 
accuracy examined in studies like these, little is known about how teachers interpret 
content coverage items or how they formulate answers to such questions. Drawing upon 
psychological theory of the survey response process, this paper investigates how 
qualitative “thinkaloud” protocols and corresponding analytic techniques can be used to 
assess how teachers interpret and respond to content coverage items. In doing so, the 
paper examines the use of qualitative techniques as tools for diagnosing the validity of 



survey items and discusses the implications of using such techniques for future surveys of 
instructional practice. 

The cognitive task of answering content coverage items 

Our theoretical and analytical perspectives in this paper are heavily colored by 
assumptions we make about the cognitive tasks required of teachers when they answer 
content coverage items. We believe that content coverage items engage teachers in a 
uniquely complex task. Such items begin with a list of topics that are thought to 
encompass some portion of a curriculum, that has been developed in a particular 
academic subject, for students of a particular age or grade. 1 For example, the OTL items 
from the Prospects Classroom Teacher Questionnaire are thought to encompass the 
language arts and the mathematics curriculum for elementary grade students. There is a 
great deal of variation in the specificity and the comprehensiveness of topic lists. 

Whereas the math topic list for the Prospects Classroom Teacher Questionnaire contains 
10 topics, including a single topic for “geometry”, the math topic list for the TIMSS 
Population I Teacher Questionnaire contains 36 topics, including five geometry topics 
such as “perimeter, area and volume”, and “congruence and similarity.” Despite the 
variation in the their length and specificity, topic lists are significantly longer than the 
four, five and six-point rating scales that are the hallmark of many questionnaires. 
Moreover, the language used to describe topics is often quite complex and replete with 
technical terms. 

This characterization reinforces the notion that answering content coverage items 
is a very complex cognitive task. Such items require teachers to formulate a summary 
judgement of considerable complexity by having them reflect back over multiple days of 



1 In charting out the tasks required of teachers in responding to content coverage items, teacher 
questionnaires from the following recent surveys of elementary school teachers were reviewed: Prospects, 



instruction, across a multitude of different lessons in which a variety of techniques were 
used with a wide array of student configurations, and then requiring them to select a 
subset of topics from a list that may contain 30 or more different choices. This paper 
attempts to unpack the ways in which teachers interpret and process through this 
complexity and to investigate how such processing affects teachers’ responses to content 
coverage items. 

Sources of evidence and analytic approach 

Data used for this paper come from interviews conducted with 12 elementary 
school teachers in a small Midwestern U.S. city. The interviews were conducted as part 
of a pilot of a self-administered questionnaire called the Instruction Log that captures 
daily reports of teachers’ instruction. This work is part of a larger program of research 
called the Study of Instructional Improvement which is a longitudinal study housed at the 
University of Michigan that will examine the improvement efforts of 125 schools over 
the next six years. 

The interviews utilized “thinkaloud” techniques, sometimes referred to as 
cognitive interviews, where respondents verbalize their thinking processes as they answer 
interviewers’ questions or questionnaire items. Thinkalouds and related techniques have 
become increasingly accepted as a useful way to improve questionnaire items by 
examining respondents’ cognitive processes (see Sudman, Bradbum, and Schwarz 1996, 
for an overview and Schwarz and Sudman, 1995 for a more in depth discussion of these 
techniques). The cognitive interviews conducted for this study have been transcribed and 
analyses were conducted by coding and systematically evaluating transcripts using QSR’s 
NVivo program. 




TIMSS, Reform Up Close, the U.S. Elementary Reading Instruction Survey, Congress to the Classroom, 
and recent studies conducted by the Consortium on Chicago School Research. 
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The interviews used for this paper engaged teachers in two tasks: 1) an 
unprompted report of topics taught in a single school day, and 2) choosing the math and 
reading topics they taught from a topic list. In the unprompted report section, 
respondents were asked to report the math and language arts topics their students worked 
on during the school day immediately prior to the interview. Participants were asked to 
“think aloud” as they recalled topics, sharing their thought processes as they remembered 
what they had taught. Teachers’ responses during this unprompted report yielded 
evidence of how teachers defined specific curricular topics. The unprompted report also 
illuminated various kinds of contextual clues teachers used to aid their recall such as 
instructional activities, routines, and materials used. 

In the choosing topics taught section of the interview, teachers were shown lists 
of math and language arts topics that had been developed for the daily Instruction Log. 
The topic lists, which are presented in Appendix A, are similar to those used in recent 
national surveys of elementary school teachers such as Prospects and TIMSS. Going 
through a record of their day they had had compiled with interviewers, teachers were 
asked to choose topics on our topic lists that corresponded to the topics contained in their 
record. Again using a thinkaloud approach, participants were encouraged to verbalize 
their thinking as they located their topics on the lists and to report any difficulties they 
encountered in making this translation. This section of the interview yielded additional 
evidence of how teachers conceive of specific curricular topics. 

In analyzing the results, the authors read each of the interviews and identified a 
number of reoccurring themes. A set of codes based on these themes was developed by 
the investigators. Transcripts that were coded according to this coding scheme served as 
the primary database that was examined for this paper. 
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A variety of techniques were used to analyze the interview data. Character and 
passage counts and paragraph markers were examined to gain a general picture of the 
relative emphasis placed on different context clues. Looking across teachers, transcript 
sections that were assigned the same code were re-read to identify sub-themes and to 
develop fuller explanations of issues that emerged. The degree of agreement between 
teachers’ and investigators’ definitions of two key reading topics was assessed and coded. 
In presenting the results below, a number of extended transcript passages are provided to 
illustrate particular themes. Methods of data manipulation and analysis are explained in 
further detail below within the context of the analyses for which they were used. 

The remainder of this paper is devoted to presenting the results of two empirical 
analyses: one which examines the contextual information on which teachers draw when 
reporting topic coverage, and another which investigates teachers’ understanding of 
reading topics. 

Contextual information on which teachers draw when reporting topic coverage 

This first set of analyses documents the contextual information and cues teachers 
draw upon to remember the content their students work on. Recalling previously taught 
topics requires teachers to access autobiographical memory. Some psychologists believe 
that autobiographical memory is partially organized in terms of meanings that are 
attached to events as they are comprehended. In recalling an autobiographical event, 
individuals search among memories that have been encoded with similar meaning. This 
model of autobiographical memory suggests that recall will be better when questions 
cause respondents to remember information in a way that taps the way in which the 
information was originally encoded. OTL items typically require teachers to report on 
their content coverage without reference to contextual clues such as the activities in 
which students engaged when they worked on the topic or the materials that were used. 



Reporting on content covered in this kind of decontextualized manner may create an 
unnatural response task that makes OTL items difficult to answer. Making reference to 
the context in which a topic was taught might in turn aid teachers’ recall. 

This first analysis primarily focuses on teachers’ interpretation of the word 
“topic”, a pivotal term which is contained in the stem of many content coverage items. In 
the unprompted topic report section of the interview teachers were asked the following 
question: 

I would like you to spend just a few brief minutes to tell me what math and what 

reading/writing topics you taught to students today. 

In responding, teachers were encouraged to thinkaloud. They did not receive any 
prompting from the interviewer and were not given a topic list as a reference point. In 
this sense, this section of the interview simulates what one might observe if a teacher 
were asked just the stem of a typical content coverage item and not presented with the 
long list of topic response choices. 

Our analysis of teachers’ transcripts indicated that explicit reports of topic 
coverage tended to be surrounded by substantial contextual information. Table 1 
illustrates that when directly asked to report the math and reading topics they had taught 
that day, most teachers did not explicitly mention a topic until well over half way through 
the unprompted report section of the interview. Perusal of the transcripts revealed that 
before they explicitly mentioned a topic, most teachers provided a rich contextual 
background about their teaching of the topic. 
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Table 1: Percent of interview that took place prior to the first explicit mention of a 
reading or mathematics topic 



Teacher 

ID 


Number of 
paragraphs in 
transcript 


Paragraph 
in which 
topic is first 
explicitly 
mentioned 


Percent of 
interview prior 
to first explicit 
topic mention 


10 


54 


11 


20% 


01 


64 


31 


48% 


04 


36 


19 


53% 


08 


12 


7 


58% 


06 


32 


19 


59% 


05 


55 


34 


62% 


12 


16 


15 


94% 


02 


20 


19 


95% 


09 


20 


19 


95% 


11 


28 


27 


96% 


03 


19 


19 


100% 


Overall 


32 


20 


62% 



When they did finally make explicit mention of a math or a reading topic, these 
mentions were relatively brief compared to teachers’ descriptions of other contextual 
factors. As Table 2 illustrates, the average teacher spent about eleven percent of the 
interview explicitly discussing content coverage. 

Table 2: Percent of transcript text (measured in text characters) devoted to explicit 
discussion of academic content 



Teacher ID 


Percent of transcript 
text devoted to 
explicit discussion 
of topic 


05 


3% 


10 


4% 


03 


5% 


09 


8% 


06 


11% 


04 


11% 


01 


12% 


12 


15% 


11 


18% 


08 


20% 


02 


29% 


Overall 


11% 
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When not explicitly discussing topics, teachers’ reports were focused on 
contextual factors that seemed to help them recall topics that were taught. In many cases 
it also seemed that recalling the contextual information was intended to help the 
interviewer understand what teachers did in their classrooms that day. Our analysis of 
teachers’ transcripts revealed five contextual factors that were particularly prominent in 
teachers’ descriptions: routines, instructional activity, students, materials, and aides or 
other teachers. The number of times each of these factors was mentioned is shown in 
Table 3. In order to portray its relative emphasis by teachers, the number of times 
academic topics were explicitly mentioned is also included in Table 3. 

Table 3: Contextual factors mentioned in topic unprompted topic reports 



Contextual factor 


Number of 
Times Factor 
Mentioned 


Percent of times 

Factor 

Mentioned 


Routines 


42 


12% 


Instructional activity 


156 


44% 


Students 


43 


12% 


Materials 


30 


9% 


Aides/other teachers 


9 


3% 


Academic content 


72 


20% 


Total 


352 


100% 



O 
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By far, the factor given the greatest prominence in teachers’ unprompted reports 
of topics was instructional activity. Instructional activity includes the academic tasks in 
which students engage and the different ways in which teachers and students interact in 
the classroom. Although instructional activity necessarily focuses on academic content, 
teachers’ descriptions of instructional activity focused mostly on what they and their 
students did, rather than what they were studying. But it is precisely because 
instructional activities always focus on content that relying on instructional activity as a 
guidepost for recalling topics taught seemed particularly useful to teachers. If teachers 
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were able to remember what they and their students did, it seemed easier for them to 
reconstruct the academic content they taught. Consider the following excerpt from 
Teacher 09’s transcript in which she talks about her topic focus in math within the 
context of the day’s activities. 

Teacher 09: And then we went to math again. We did a “Mad Minute,” fifty 
multiplication problems. I just yell out the times, one minute, two minutes, three 
minutes, so kids can time themselves and figure out what they can do to challenge 
themselves, if they need to get down to three minutes or down to two minutes or 
down to one minute, but everybody finishes the problems, and then we just 
flipped the paper over and worked on a few multiplication problems, two by one, 
two digits by one digit. ...And then today’s computational math problems, 
multiplication, division and adding , subtracting multiplying and dividing 
fractions. And we worked on those. A small group of kids came up to the board, 
and we worked on dividing fractions for a while. 

Students and materials also seemed to provide useful guideposts for teachers. 
Recalling what they did with particular students or what materials they were using 
seemed to help some teachers reconstruct what academic content they focused on during 
the day. 

Teacher 02: Okay, they had a choice time at the beginning of the day and there 
was a bingo game out that had sight boards, so some of them were working on 
sight boards. Some of them were drawing and writing of their own choice. Some 
were doing activities on the computer and whatever they choose to do on the 
computer in the morning, they have free choice. Others were using some Geo 
blocks and building cities and whatever they happened to choose to build; 

A considerable number of teachers relied upon instructional routines to aid their 

recall. It is well documented that teachers use a variety of routines, that occur with 

varying periodicity, to organize classroom instruction. In a number of cases, teachers 

appeared to use routines as markers to initially orient themselves. For example, this 

teacher began her report of topic coverage with a description of reading instruction on a 

typical day. 

Interviewer: First of all, what I’d like to do is just have you take a few minutes to 
talk to me about what kind of math and reading and writing topics the students 
worked on today. 
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Teacher 04: Today? Okay. In terms of reading, I do have a regular reading time, 
so that’s easier to remember, I’ll start with that. Right after lunch is an 
independent reading time, and they all read individually, and I often read... that’s 
when I do most of my reading instruction. I’ll take one or two kids and read with 
them one to one. And while I read with them one to one, it varies a lot as to what 
they’re working on. . . .Let’s see, as I’m reading, so many of them are reading 
quite well now, so we hit, we might hit a little bit of phonics as we’re reading 
along, but mostly we just do a lot of reading for general meaning and trying to put 
it all together. 

Once oriented by their routines, it seemed that a number of teachers could more 

easily establish what happened on the day we interviewed them. 

Teacher 01: Ok... So, everyday we have a snack and story time. And the children 
share a snack together and we read out loud to them from a story often related to a 
theme, in this case to our mathematics. We’re doing some geometry, and we have 
a story ah. . . about a quilt that we read out loud. And so the topic at that point, 
now I have to start thinking about your reading topics in your list. . . I read out 
loud to them everyday, and for a variety of purposes. In this case it was to 
enhance the theme, its practice that.. . for them for listening, its vocabulary 
development. 

The results of this first set of analyses suggest two things: 1) that when recalling 
topics, teachers seem to pay attention to features of classroom life that are salient to them 
and that help orient them, 2) academic content does not appear to be a primary organizer 
for many teachers’ conceptions of their instruction. These results suggests that content 
coverage items might be improved if survey designers gave greater consideration to the 
ways in which teachers appear to think about academic content. 

Teachers’ understanding of reading topics 

The second set of analyses dealt with teachers’ understanding of two key topics in 
the reading topic list. Our goal here was not to validate every topic on the lists, but 
rather, to probe the efficacy of different techniques that survey researchers might use to 
more generally validate respondents’ understanding of questionnaire items. 

Methodologists who study how individuals respond to survey items generally 
agree that the first step in answering an item is to understand its meaning. Research has 
shown that “lexical ambiguity”, which occurs when words potentially take on more than 



one meaning for a respondent, is a common problem in question comprehension. Lexical 
ambiguities are inherent in everyday language. One reason for this is that alternative 
meanings of the same word may be differentially accessible to different people because 
of the frequency with which they use or encounter the word. Lexical ambiguity can also 
result when words take on different meanings within different groups and subcultures. 
Ambiguous meaning is a clear threat to valid measurement since the validity of a 
response is dependent on respondents sharing researchers’ understanding of an item’s 
meaning. 

Because of their heavy reliance upon technical terms, content coverage items may 
be particularly prone to problems associated with lexical ambiguity. One potential 
problem is that teachers may simply be unfamiliar with terms used in topic lists. This 
conjecture has some grounding in empirical results as Burstein et al (1995) found a 
number of teachers did not understand what “proportional reasoning” and “math 
modeling” meant even though these topics appeared in key math reform documents to 
which teachers’ had likely been exposed. Another common problem, also observed by 
Burstein et al (1995) arises when a topic can be thought of both as abstract knowledge 
within a content domain and as a strategy that can be used to learn abstract knowledge 
within a domain. For example, “making inferences from data” can either be the 
substantive focus of a math lesson or can be a strategy that students use to work on other 
mathematical topics. 

The validity of teachers’ topic reports was assessed by comparing teachers’ 

understanding of two key reading topics with those of the investigators. Following are the 

investigators’ definitions of the two topics: 

Sight words (topic 1.2.03): Sight words are words that students are to quickly 
recognize and read without needing to decode them. They typically include very 
common words (i.e. then, it, help), or words that are very difficult to sound out 



using conventional phonetic rules (i.e. have, the, some). Typical ways in which 
teachers work with sight words are: word walls, flash cards, and word lists. 

Word reading strategies (topic 1.2.06): This topic encompasses a number of 
specific strategies that students use to read words including: the use of pictures or 
other context cues, and paying attention to specific characteristics of words such 
as the initial consonant, rhyming, and common word endings. 

The analyses were conducted in two steps. First, teachers’ conceptions of the two 

key reading topics were ascertained by analyzing teachers’ descriptions of their reading 

instruction for a single day. These descriptions were gleaned from transcripts of the 

unprompted report section, and teachers’ topic reports were taken from the choosing 

topics taught section of the interview. In the second step of the analysis, teachers’ 

conceptions of the topics were compared to those of the investigators and each topic 

report was placed into one of the following categories: 

• Teachers’ conception is consistent with researchers’. Topic reports were 
assigned to this category when teachers’ descriptions of classroom activity or 
content covered indicated their understanding of the topic was the same or very 
similar to the above topic definitions. 

• Teachers’ conception is inconsistent with researchers’ . Topic reports were 
assigned to this category when teachers’ descriptions of classroom activity or 
content covered indicated that their understanding of a topic differed substantially 
from the topic definitions listed above. 

• Unable to determine teachers’ conception of topic. Topic reports were assigned 
to this category when teachers’ description of classroom activity or content 
covered provided inconclusive evidence about whether the teacher shared the 
researchers’ understanding of a topic. 

Overall, only one teacher expressed an understanding of a topic that was clearly 
different from the investigators’ definition. The degree to which teachers and researchers 
possessed the same understandings varied by topic. Teachers’ descriptions of instruction 
that focused on sight words were highly consistent with the investigators’ definition of 
that topic. In comparison, teachers’ reports of teaching word reading strategies were 
more equivocal, and thus, it was more difficult to judge the degree to which teachers 
shared researchers’ definition of that topic. The results of the analysis of teachers’ 



descriptions of teaching sight words and word reading strategies are summarized in 
Table 4. Excerpts from interview transcripts, which detail teachers’ understanding of the 
two topics, and justifications for the classification of each teachers’ topic reports can be 
found in Appendix B. 

Table 4: Summary of the validation of teachers’ reports of two reading topics 



Topic 


Teachers’ conception 
was consistent with 
researchers’ 


Teachers conception 
was not consistent 
with researchers’ 


Unable to determine 
teachers’ conception 
of topic 


Sight words 


6 


0 


2 


Word reading 
strategies 


3 


1 


6 



Sight words 

Among the twelve teachers who reported on a single day of their instruction, the 
topic sight words was mentioned a total of eight times. In six of the eight cases, teachers’ 
descriptions of their instruction indicated that their conception of the sight words topic 
was consistent with the investigators’ definition. Consistent reports were associated with 
a variety of instructional activities at a number of grade levels. For example, a K-l 
teacher worked with her students on sight words that were contained in books they had 
chosen themselves: 

Teacher 01: Today. . . we read from trade books, so they’re choosing books. I 
have books that are graded from pre-primers through chapter books. Today, I 
worked with two or three kids that are still just getting the first like hundred sight 
words... So these are books... that have 64 words in them. I pull them out for the 
kids that don’t seem to think they can read books, but are learning all the sight 
words and everything else but that. 

Another primary-grade teacher had her students work on sight words while they played a 
game: 




Teacher 02: And after lunch, they were working with sight words', they were 
playing some games with them. We play a game where there are words, words 
like “the sun” located on my wall too, and we play a game where “Guess my 
word?” and I’ll say something like “Well, it is on the wall” and “Well, it begins 
with the letter ‘L’ ” and I give them clues until they can finally come up with the 
word. 
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One of the more interesting reports came from a third grade teacher who was not 
working on sight words with her own students, but whose students were helping to teach 
kindergarten students how to read. The teacher reported that she trained her students how 
to work on sight words and various other reading skills with the kindergarten students. 

In two cases, teachers simply reported that they worked on sight words with their 
students but did not provide any further elaboration of how the topic was taught. This 
meant that we were unable to determine whether the teachers’ conception of sight words 
aligned with our own. As we discuss below, many teachers’ reports of word reading 
strategies also lacked this kind of descriptive detail. 

Word reading strategies 

Work on word reading strategies was mentioned a total of ten times in the 

cognitive interviews. In three of the ten cases, teachers’ descriptions of their instruction 

indicated that their understanding of word reading strategies was closely aligned with the 

investigators’ definition. Perhaps the clearest example of this is the third grade teacher 

whose students helped teach kindergarten students how to read. 2 

Teacher 12: My children were working with the kindergartners. We had given 
them a sort of mini-training so they were working with the kindergartners on how 
to figure out what a word is by looking at the picture. Not only by looking at the 
picture, but context clues and hints, and beginning letter sounds so a lot of these 
subcategories they were working not necessarily for themselves but to help the 
kindergartners pick it up. So I’d say print concepts 1.1.01. 1.1.02. 1.1.03. 1.2.01. 
1.2.02. 1.2.03 sight words. Oh, wait. 1.2.06 word reading strategies like picture 
and context clues so they were pointing to pictures and having the kindergartners 
say “bicycle” so the word is “bicycle.” 

The degree to which teachers’ understanding of this topic overlapped with our 
definition could not be determined in six of the ten cases. In nearly all of these cases, 
teachers claimed to work on word reading strategies with their students but did not 



2 It is interesting to note that under our current data collection plans for the Instruction Log , teacher 09’s 
report of working on sight words and word reading strategies would be in error, because we currently ask 
teachers to report on specific students in their own classes. Since teacher 09’s third graders were not 
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provide a sufficient description of their teaching to validate their claim. In many 
instances, teachers simply reported that they worked on the topic and said no more about 
the matter. However, the reports of a few teachers went a bit further because they 
included activities that could plausibly be associated with the coverage of word reading 
strategies. For example, teacher 05’s report suggested that she was assessing students’ 
ability to use word reading strategies in order to determine the appropriateness of books 
these students had chosen. 

Interviewer: What about the four children that you were reading with... 

Teacher 05: That I read with individually? I was looking for, I was looking for 
1.2.02, 1 was looking for 1.2.03 and 1.2.06 in other words, if they wanted to 
continue reading the same kind of book we were reading, I need to make a 
decision if they were only going to become more frustrated or if I had to change 
what they were choosing. 

One might reasonably infer from this scenario that teacher 05 covered word 
reading strategies. Despite this plausibility however, there was insufficient detail in 
teacher 05’s transcript to determine whether she in fact shared our definition of this topic. 

This analysis of teachers’ reports of word reading strategies also revealed a topic 
omission. Teacher 02’s interview clearly indicated that she worked on word reading 
strategies with her students but she did not report this topic. In order to fully capture 
omissions like this, one would need a different research design than that used for this 
study. Within the confines of the larger study of which this work is a part, we are 
undertaking a more comprehensive investigation in which the topic reports of 
approximately 30 teachers will be validated against classroom observations. 

The techniques used for the second set of analyses seemed relatively useful in 
surfacing respondents’ understanding of item response choices and in suggesting ways in 
which we might want to clarify our topic definitions. Despite our very limited scope, the 




actually working on these two topics, her report would have been erroneous. This is an example of the 
kinds of issues that can be revealed when qualitative techniques are used to diagnose questionnaires. 
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evidence suggested that we can be quite confident that we understand what teachers did 
when they reported to us that they worked on sight words. The results for word reading 
strategies were much more equivocal however. We suspect that our results might reflect 
the ways in which the topics were defined. While there are many different ways to work 
on sight words, the topic is relatively unidimensional. That is not the case for word 
reading strategies which includes a number of different approaches to reading. A simple 
assertion that one worked on sight words is a fairly unambiguous (albeit general) 
indicator of what kind of instruction is taking place. A similar kind of assertion that word 
reading strategies took place is much more ambiguous as there are many more things 
wrapped up in that topic. More explicitly specifying the approaches that are included in 
word reading strategies might lead to less ambiguous responses. 

The results also surfaced potential limitations of the interview protocol. Once 
teachers had chosen the topics they had taught from the lists, we could have probed for 
more explicit descriptions of each topic chosen. This might have helped resolve some of 
the ambiguity in teachers’ reports of word reading strategies. 

Discussion 

Survey researchers often take for granted what respondents intend when they 
answer a question. In the case of student OTL items, one might assume that the topics 
one teaches are quite easily retrieved in a direct fashion. The results presented in this 
paper suggest that this is not the case, that instead, teachers are likely to draw on a rich 
set of contextual information when recalling what they covered. Without any stimuli to 
guide them, teachers relied on a variety of internally-generated guideposts to aid their 
recall. The presentation of topic lists seemed to focus teachers’ thinking on content. One 
clear implication of this is that researchers interested in measuring content coverage 
would be highly advised to stick with topic lists, despite their limitations. It was clear 



that presenting teachers with the stimulus of the list triggered teachers to think about the 
content they taught, whereas the open-ended nature of the unprompted report triggered 
the recall of a lot of other kinds of information. We would extend a more general 
cautionary note to survey researchers who are contemplating the use of open-ended 
questions. 

Our findings further suggest that teachers’ understanding of the topics in topic 
lists can not be taken for granted, and that if they really wish to be confident about the 
validity of teachers’ responses to such items, researchers must explicitly investigate such 
matters. Opportunity for differences in understanding are particularly acute when the 
domain being measured is large, as it was in our case. While not a panacea, our findings 
suggest that investigating respondents’ descriptions of what gets coded into response 
choices can affirm the validity of responses and highlight a variety of response problems. 

The results of this research have also pushed us to think about how we might 
modify items to make them clearer and more intuitive to respondents. One avenue of 
improvement suggested by the results would be to intersperse questions that ask about 
instructional activity or some of the other guideposts used by teachers with content 
coverage items. Sudman et al (1996) suggest that the content of preceding questions can 
increase the accessibility of a concept in memory thus increasing the likelihood that this 
concept rather than another is used in resolving perceived ambiguities in a question’s 
meaning. This suggests that it might be possible to contextualize content coverage items 
by preceding them with questions about instructional activity, routines, students, or 
materials. 

We feel that the techniques described in this paper helped us understand our 
instrument better and believe these techniques may be a useful addition to the toolkits of 
survey researchers who wish to better understand the meaning of respondents’ answers. 



Working in this direction strikes us as an important matter since the meaning of survey 
results and the validity of inferences one can make from such results hinges upon 
respondent and investigator sharing an understanding of a question’s meaning. 
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Appendix A: Reading and Math Topic Lists 

1. Reading Topics. 

7.7 Pre-reading. 

1.1.01 Print concepts (use and 

organization of text and 
books). 

1.1.02 Phonemic awareness (hearing 

sounds, recognizing rhymes) 

1.1.03 Language concepts (purpose of 

text: enjoyment, information) 

7.2. Letter and word analysis / reading 

1.2. 01 Letters/ sounds 

1.2. 02 Phonemic knowledge and skill 
(sound out letter combinations 
and words, write letter 
combinations and words from 
sounds/ phonetic spelling) 

1.2. 03 Sight words (recognize whole 
words) 

1.2. 04 Conventional spelling 

1.2. 05 Structural analysis of words 
(compound words, suffixes, 
roots, word families, 
syllabification) 

1.2. 06 Word reading strategies (picture 

and context clues, initial 
consonant, common endings, 
compound words, read-stop- 
reread) 

7.5. Reading comprehension 

1.3. 01 Word meaning (vocabulary 

knowledge, relationship of 
word meanings— antonyms, 
synonyms, multiple meanings, 
etymologies) 

1.3. 02 General comprehension 

strategies at the sentence and 
paragraph level (strategies for 
activating prior knowledge/ 
generating predictions; 
question answering strategies; 
strategies for checking 
comprehension) 

1.3. 03 Structural analysis of text 

(identify main idea, character; 
recognize language patterns, 
rhyme) 

1.3. 04 Literary conventions (imagery, 
symbolism, metaphor, point of 
view, fact/opinion, mood/tone, 
reality/fantasy) 

1.3. 05 Literary forms (report, literary 
response, narrative, poetry, 
biography, fantasy, historical 
fiction) 

1.3. 06 Read for a variety of purposes 
(for enjoyment, to learn content, 
to explore a question of interest) 



1.3. 07 Listened for a variety of 

purposes (for enjoyment, to learn 
content, to explore a question of 
interest) 

1.4. Written composition 

1.4. 01 Grammatical and mechanical 

conventions (formation of 
letters, spacing between words, 
complete sentences, syntax) 

1.4. 02 Beginning composition at the 
word and sentence level 
(understand that writing 
conveys meaning; label 
objects, ideas, feelings with 
words or sentences) 

1.4. 03 Select topic and focus, generate 
and organize ideas. 

1.4. 04 Identify audience, target writing 
to audience. 

1.4. 05 Word choice (choose words to 
persuade or entertain; chose 
words for different written 
conventions and forms) 

1.4. 06 Select and develop structural 
features of text (main idea, 
character development, 
language patterns, rhyme) 

1.4. 07 Select and develop literary 
conventions (imagery, 
symbolism, metaphor, point of 
view; fact/opinion, mood/tone, 
reality/fantasy) 

1.4. 08 Select and develop of literary 
forms (report, literary 
response, narrative, poetry, 
biography, fantasy, historical 
fiction) 

1.4. 09 Proofread and edit written 
composition (spelling, 
punctuation, grammar; word 
choice, structural features of 
text, writing conventions) 

1.4. 10 Student wrote for a variety of 
purposes (for enjoyment, to 
learn content, to explore a 
question of interest). 

7.5. Study skills (use of dictionary, table of 
contents, glossary, index, encyclopedia). 
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Appendix A (continued) 

2. Math Topics 

2.7. Number and Operations 

2.1. 01 Counting and number sense 

2.1. 02 Read and recognize numbers 

2.1. 03 Sets and classification 

2.1. 04 Comparing, ordering 

2.1. 05 Number concepts (e.g., even, odd, 
prime, composite) 

2.1. 06 Grouping and place value 
2.1.07 Numeration 

2. 1 . 08 Integers 

2. 1 . 09 Fractions 

2.1. 10 Decimals 

2.1. 11 Addition concepts, basic 
combinations 

2.1. 12 Subtraction concepts, basic 
combinations 

2.1. 13 Multiplication concepts, basic 
combinations 

2.1. 14 Division concepts, basic 
combinations 

2.1.15 Addition computation 

2.1.16 Subtraction computation 

2.1.17 Multiplication computation 

2.1. 18 Division computation 

2.1. 19 Factors, multiples, divisibility 

2.1.20 Ratio, proportion, percent 

2.1.21 Real numbers 

2.2 Geometry 

2.2. 01 Points, rays, lines, segments, 

planes 

2.2. 02 Angles 

2.2. 04 Symmetry 

2.2. 05 Visualization, spatial reasoning 

2.2. 06 Parallel, perpendicular 

2.2 .07 Polygons (triangles, squares and 
rectangles, other quadrilaterals, 
other) 

2.2. 08 Circles 

2.2. 09 Coordinate geometry 

2.2 .10 Similarity, congruence 

2.2. 11 Symmetry 

2.2. 12 Solid figures (spheres, pyramids, 
polyhedra) 

2.2. 13 Transformations 

2.2. 14 Pythagorean theorem 

2.3. Measurement 

2.3. 01 Non-standard measure 

2.3. 02 Calendar 

2.3 .03 Time 

2.3. 04 Length 

2.3 05 Perimeter 

2.3. 06 Area 

2.3. 07 Volume, capacity 

2.3. 08 Angle 

2.3 09 Weight 

2.3. 10 Temperature 

2.3. 11 Rates 

2.3. 12 Convert measurement units 

2.3. 13 Circumference 
24 Probability 



2.4 .01 Events, possible outcomes 

2.4 .02 Equally likely 

2.4 .03 Empirical probabilities 

2.4 .04 Calculation of theoretical 

probabilities 

2.5 Statistics 

2.5. 01 Collecting and organizing data 

2.5. 02 Mean, median, mode 

2.5. 03 Sampling 

2.5. 04 Draw conclusions from data 

2.5. 05 Evaluate conclusions drawn from 
data 

2.5. 06 Make inferences from data 

2.5. 07 Describe, evaluate data 

2.5. 08 Select data display 

2.5. 09 Complete/construct data display 

2.5. 10 Interpret data display 

2.5. 11 Compare data 

2.5. 12 Use data to solve problem 

2.6 Patterns , Functions, and Algebra 

2.6. 01 Patterns 

2.6 . 02 Missing elements 

2.6. 03 Variables 

2.6 . 04 Functions 

2.6 . 05 Equations 

2.6. 06 Inequalities 

2.6. 07 Graph linear problems - 

2.6. 08 Use algebra to solve problems 

2. 7. Exploration and problem solving 

2.7. 01 Formulate problem 

2.7. 02 Spatial reasoning 

2.7. 03 Proportional reasoning 

2.7. 04 Solve non-routine problem 

2.7. 05 Deductive/inductive reasoning 

2.7. 06 Identify missing/extra 
information 

2.7. 07 Evaluate solution 

2.7. 08 Evaluate conjectures 

2.7. 09 Develop and explain strategy 
28 Reasoning, Proof 

2.8 .01 Investigate conjectures 

2.8 .02 Develop/evaluate mathematical 
arguments and proofs 

2.8 .03 Proving completeness of a 

solution 

2.8. 04 Select/use various types of 

reasoning and methods of proofs 

2.9 Communication 

2.9 .01 Using and creating 

representations (drawings, 
graphs, concrete objects( 

2.9 .02 Symbolic notation 

2.9 .03 Using language to talk and write 
about mathematical ideas 
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Appendix B: Analysis of teachers’ understanding of the reading topics sight words and 
word reading strategies 



Teacher 


Evidence of Teachers’ Conceptions of Topics 


Analysis 


Teacher 


Teacher: Today. . . we read from trade books, so they’re 


Sight words 


01, grades 


choosing books. I have books that are graded from pre- 


( 1.2.03). Teachers’ 


K & 1 


primers through chapter books. [Students] independently 


conception is 




choose unless I have a particular task for them in mind that 


consistent with 




day. Today, I worked with two or three kids that are still just 


researchers’. The 




getting the first like hundred sight words. . . They’ve been 


fact that the teacher 




reading for confidence some limited vocabulary story books. 


explicitly 




In particular as I remember today, [they read] Margaret 


mentioned that she 




Helert stories. So these are books. . . that have 64 words in 


had students read 




them. I pull them out for the kids that don’t seem to think 


“limited vocabulary 




they can read books, but are learning all the sight words and 


books” so that they 




everything else but that. 


would learn sight 
words is fairly 




Teacher: So the group of kids. . . that were reading from the 


strong evidence that 




limited vocabulary books, I don’t know if we want to call that 


she shared 




sight words. My goal wasn’t necessarily for them to practice 


researchers’ 




the sight words as much as it was for their attitude about 


conception of this 




books as we talked about already, and their sense of. . . “Oh, I 
can do the whole book!” But they did that by practicing the 
sight words that there in these books. So there is a topic that 


topic. 




says sight words. . . 


Word reading 
strategies ( 1.2.06). 




Interviewer. So I’ll put that down. 1 .2.03 Ok. . . 


Unable to 

determine teachers’ 




Teacher: This is one of the one’s too that we were talking 


conception of topic. 




about, word reading strategies. . . .integrating different 


The teacher said she 




strategies, ... 1 .2.06 was certainly [a focus] for the kids that 


focused on reading 




were reading from the other trade books. . . 


strategies and 
integrated different 




Interviewer: Do you think that both [topics 1.2.03 and 1.2.06] 


strategies but did 




apply to the students in the limited vocabulary? Or is this the 


not mention nor 




[only] one you would choose... 


describe specific 
strategies. This is 




Teacher: I think they would both apply in the limited 


suggestive, but not 




vocabulary books. 


clear evidence that 
the teacher might 
share researchers’ 
conception of the 
topic. 



Teacher 


Evidence of Teachers’ Conceptions of Topics 


Analysis 


Teacher 
02, grades 
K & 1 


Teacher: Okay, they had a choice time at the beginning of the 
day and there was a bingo game out that had sight words, so 
some of them were working on sight words. 

Interviewer: Okay, well, turning this over, [this first 
activity], it is just sort of like a bingo game. It is like words. 

Teacher: Yeah, sight words, 1.2.03 is what would go with 
that. 


Sight words 
( 1.2.03). Teachers’ 
conception is 
consistent with 
researchers’. There 
is strong evidence 
that the teacher 
shares researchers’ 
conception of the 
topic. The teacher 
explicitly 
mentioned that 
students worked on 
sight words as they 
played bingo. 
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Teacher 


Evidence of Teachers’ Conceptions of Topics 


Analysis 


Teacher 
02, grades 
K & 1 


Teacher: And after lunch, they were working with sight 
words', they were playing some games with them. We play a 
game where there are words, words like “the sun” located on 
my wall too, and we play a game where “Guess my word?” 
and I’ll say something like “Well, it is on the wall” and 
“Well, it begins with the letter ‘L’ ” and I give them clues 
until they can finally come up with the word. They get five 
clues until they can figure it out, so we spent some time doing 
that. And then they write the word down, and we say it back 
and chant it back so we have a couple different ways to learn 
what that word is. 

Interviewer: Okay, we just have a few more here, and there 
was the sight words, guessing words. 

Teacher: Okay, that is what it is, is sight words recognition. 
Interviewer: Okay, 1.2.03 


Sieht words 
(T. 2.03). Teachers’ 
conception is 
consistent with 
researchers’ . There 
is strong evidence 
that the teacher 
shares researchers’ 
conception of the 
topic. The teacher 
explicitly 
mentioned that 
students worked on 
sight words as they 
played the “Guess 
my word” game. 

Word readme 
strateeies (1.2.06). 
Unable to 

determine teachers ’ 
understanding of 
topic. The teacher 
described an 
activity which 
directs students to 
focus on initial 
consonants when 
reading words. 
Despite this 
description, the 
teacher did not 
report that this topic 
was taught. It is 
impossible to tell 
whether she simply 
omitted this topic 
from her topic 
reports or whether 
she did not share 
researchers’ 
description of the 
topic label. 
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Teacher 


Evidence of Teachers’ Conceptions of Topics 


Analysis 


Teacher 
03, grades 
2 & 3 


Teacher: And so we are involved with the University of 
Chicago’s junior Greek book series which takes unabridged 
collected works by Rudyard Kipling and Hans Christian 
Andersen and so they are some very advanced reading series. 
And they talked about Jack and the Beanstalk and the 
political ramifications of when that story was written and that 
was a read aloud in a small group. And then the third graders 
were reading “How the Camel Got his Hump,” and talked 
about when that was written and that was also a read aloud 
and was very insightful for me to hear that they were 
understanding that people were constructing stories and how 
stories were constructed and the multiple layers of how a 
story might be constructed. We do that for an hour each day 
so the third graders have an hour of literature and then the 
second graders have an hour of literature and that’s daily. 

Interviewer: And how about the reading series with Jack and 
the Beanstalk? 

Teacher: Okay. 1.1.02, 1.1.03, 1.2.01. 

Interviewer: We’re on the reading series right now? 

Teacher: Yes, isn’t that right? 1.20.3. 1.2.06. 1.3.01. 

Sorry. 1.3.02. 1.3.03. 1.3.04. 1.3.07. 1.3.06. Sorry I 
missed that one. Okay, that’s it. 

Interviewer: Okay. And going on to How the Camel Got His 
Hump? 

Teacher: That would be the same. 


Sieht words 
fl. 2.03). Unable to 
determine teachers ’ 
understanding of 
topic 

Word reading 
strategies ( 1.2.06). 
Unable to 

determine teachers ’ 
understanding of 
topic 

The teachers’ 
description of 
classroom activity 
and content covered 
does give any 
indication about her 
conception of sight 
words or word 
reading strategies. 
She simply states 
that these topics 
were objectives of 
her reading lesson 
but did not describe 
how the topics were 
taught. 
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Teacher 


Evidence of Teachers’ Conceptions of Topics 


Analysis 


Teacher 
04, grades 
K & 1 


Teacher: Another regular thing we do in the morning is our 
morning message. We might do phonics there, patterns in 
words, I’m getting them to look at lately - but that is a whole 
group activity, and they come up individually if the want to 
show us something they know on the morning message. And 
it’s just like two lines I’ve written. It also gives them an idea 
of what we’re going to do that morning, so it’s kind of a 
preparation. 

Interviewer: Then you said you had the morning message 
where you were looking for patterns and words and things 
like that, so where would you say that would fit under? 

Teacher: Oh, a lot of different things, probably word reading 
strategies, 1.2.06. That covers quite a bit of things. 


Word reading 
strategies (T. 2.06) 
Teachers ’ 
conception is 
consistent with 
researchers ’ . The 
teachers’ report that 
she had students 
look for “patterns in 
words” is 
suggestive that she 
might share 
researchers’ 
conception of word 
reading strategies. 
In order to more 
definitively 
establish a shared 
conception, we 
would need to know 
more about what 
the teacher means 
by “patterns in 
words.” 
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Teacher 


Evidence of Teachers’ Conceptions of Topics 


Analysis 


Teacher 
04, grades 
K & 1 


Teacher: Right after lunch is an independent reading time, 
and they all read individually. I’ll take one or two kids and 
read with them one to one. I have one boy who’s really... I 
read with him more often because he’s having a hard time. 
He’s working on one-to-one correspondence words, you 
know, saying a word, because he understands getting 
meaning from pictures, and he’s got the meaning part of it 
down, now he’s sort of working on the visual and reading 
what is actually there. And he also then went on to work on 
the computer and this was his idea, to type in the book in the 
computer - you know, it’s a real short book - but he worked 
on that again, he’d been working on it yesterday too, because 
he’s very motivated to do it because it was his idea and he 
knows... he likes the book, and this is going to help him. And 
I just happened to have a copy of pictures from the book 
today too, so he was able to put them together today after 
reading time. 

Interviewer: So, the first thing I have is the independent 
reading time. So, I’m wondering where you would put that in. 

Teacher: Um, now each child had a little bit different 
purpose, cause some were working more on their strategies 
and others were reading for a variety of purposes. Put 1.2.06 
and then put 3.06. 


Word readme 
strateeies ( 1.2.06) 
Teachers’ 
conception is 
consistent with 
researchers ’. The 
teachers’ report of 
working with a 
student in using 
pictures as context 
clues suggests that 
her conception of 
word reading 
strategies aligns 
with researchers’ 
conception. 
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Teacher 


Evidence of Teachers’ Conceptions of Topics 


Analysis 


Teacher 
05, grades 
K, 1 & 2 


Teacher: Then we had half an hour of SSR *** this morning 
also, and that gave me an opportunity to read with about 4 
children because I take that time to do book exchange. And 
what I have is a home lending program, so the children can 
take books they’ve selected from my classroom. . . .parents 
are very perplexed about why children are bringing home 
books that they can’t read. . . So that gives me an opportunity 
during quiet reading to read with them. That’s usually about 
four kids per day. 

Interviewer: What about the four children that you were 
reading with... 

Teacher: That I read with individually? I was looking for, I 
was looking for 1.2.02, 1 was looking for 1.2.03 and 1.2.06 in 
other words, if they wanted to continue reading the same kind 
of book we were reading, I need to make a decision if they 
were only going to become more frustrated or if I had to 
change what they were choosing. 


Sight words 
(1.2.03) 

Word reading 
strategies (1.2.06). 
Unable to 

determine teachers’ 
understanding of 
topic. Teacher 05’s 
description suggests 
that she was 
“looking for” 
students’ 

knowledge of sight 
words and their 
ability to use 
various reading 
strategies as she 
listened to them 
read. However, 
without a more 
explicit description, 
it is difficult to 
ascertain whether 
this was in fact 
what she did. 
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Teacher 


Evidence of Teachers’ Conceptions of Topics 


Analysis 


Teacher 
06, grades 
3 & 4 


Teacher: Yeah. Now I have two or three kids who go to 
special ed., but I’m not including them on here. Usually they 
go to special ed. during this independent language arts time 
so they are doing the same things. Instead of doing reading 
and writing, they are learning to read, learning to spell. You 
know, the special ed. program is more of a one-to-one 
specialized thing. 

Interviewer: And for your two special ed. students, what 
would you say the purpose of what they were doing was? 

Teacher: I think they are doing structural analysis. And I 
think they are doing these two things. 

Interviewer: 1.2.05 and .06. 

[does teacher confirm this] ? 

Teacher: Because this sounds to me like they are really 
practicing reading words. One of them is a little bit ahead of 
the other, but I think this is what they go over. Maybe sight 
words. 


Sisht words 
(1.2.03). Teachers’ 
conception is 
consistent with 
researchers ’. 
Teacher 06’s report 
that she worked on 
sight words with 
these 2 students 
seems fairly 
unambiguous. 

Word reading 
strategies (1.2.06). 
Unable to 

determine teachers’ 
understanding of 
topic. 
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Teacher 


Evidence of Teachers’ Conceptions of Topics 


Analysis 


Teacher 


Teacher: We’ve been working with a computer software 


Sight words 


08, grades 


program created by Tom Snyder productions called 


( 1.2.03). Teachers’ 


K-5 


Neighborhood Map Machine. Basically, you can create maps 


conception is 




with it. But the part that has been very engaging for kids is a 


consistent with 




part called mystery, where you basically hide some kind of 


researchers ’ - 




object in the map somewhere, and they have to go through a 


suggestive. That is, 




series of clues to find it. So, the reading part, they have to 


if students had to 




read the clues. And that’s been a problem with 1st and 2 nd 


read words to play 




grade because some of the language is tough for them, but we 


computer game and 




do have the option with the program to record voice, and I’m 


teacher says sight 




in the process of doing that now, so all the written clues will 


words were 




have an audio part as well, so it’s fun. 


covered, there’s a 
pretty good chance 




Interviewer: Here I’d like to start with the clues, the reading 


that students 




of the clues in mystery and where you think that would fall in 


worked on sight 




here. 


words while 
playing the 




Teacher: 1.2.03, 1.2.06 is a possibility, 1.3.01, 1 guess, 

1.3.06, the variety of purposes... you have to read for and 
listen for, and some of them too do come with audio prompts, 


computer game. 




so I guess 1.3.07 would fit in there. Probably most of it, 


Word reading 




some of these are general enough that if I really wanted to 


strategies (1.2.06). 




stretch it I probably could add another... 


Unable to 

determine teachers’ 
understanding of 
topic. 
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Teacher 


Evidence of Teachers’ Conceptions of Topics 


Analysis 


Teacher 
08, grades 
K-5 


Teacher: There’s a group of 5th graders at 2:50 to 3:30 that 
[were] doing some research on states, specific states in the 
US, and they are going to do a computer presentation 
program with it. 

Interviewer: So you’d say this is a reading activity? 

Teacher: That’s a reading activity. Yeah, that’s kind of a 
short term project 

Interviewer: How about when you had the group of students 
who were working on the research on the states, and where 
do you think the topics that you’d have in there? 

Teacher: 1.2.06, 1.3.02, 1.3.06, course they’re doing a little 
writing too... do you want me to go on with this written part? 


Word reading 
strategies (T.2.06). 

Teachers’ 
conception is 
inconsistent with 
researchers’. It 
seems unlikely that 
fifth grade students 
would work on 
word reading 
strategies as this 
really is a focal area 
for early readers. 
Therefore it seems 
likely that the 
teacher had 
something different 
in mind that the 
researchers’ 
conception when 
she reported this 
topic. 
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Teacher 


Evidence of Teachers’ Conceptions of Topics 


Analysis 


Teacher 
11, grade 
3 


Teacher: And then being the kind of community this school 
is, we have a kindergarten class who comes in every 
Wednesday for reading buddies and my kids have a partner in 
kindergarten and they helped listen to the kindergartners read. 

Interviewer: And thinking about the reading buddies, let’s 
pick out the topics that are appropriate. 

Teacher: Pre-reading comes in, all these subcategories under 
1.1 and 1.2. My children were working with the 
kindergartners. We had given them a sort of mini-training so 
they were working with the kindergartners on how to figure 
out what a word is by looking at the picture. Not only by 
looking at the picture, but context clues and hints, and 
beginning letter sounds so a lot of these subcategories they 
were working not necessarily for themselves but to help the 
kindergartners pick it up. So I’d say print concepts 1.1.01. 
1.1.02. 1.1.03. 1.2.01. 1.2.02. 1.2.03 sight words. Not this 
one because I don’t think it had much to do with conventional 
spelling so we’ll skip .04, and not with the kindergartners so 
I’d stop there. Oh, wait. 1.2.06 word reading strategies like 
picture and context clues so they were pointing to pictures 
and having the kindergartners say “bicycle” so the word is 
“bicycle.” So it was with picture word. 

Interviewer: So 1.2.06? That’s helpful. I’m glad we caught 
that. 

Teacher: It wasn’t for themselves they were learning it, but I 
think that especially some of my lower readers who are 
frustrated and who think they can’t read, when they see how • 
they were teaching the kindergartners, it helps them too. 


Sight words 
Q. 2.03). Teachers’ 
conception is 
consistent with 
researchers 
Teacher 1 1 said 
explicitly that the 
kindergarten 
students worked on 
sight words. 

Word reading 
strategies (1.2.06). 
Teachers ’ 
conception is 
consistent with 
researchers’. The 
fact that teacher 1 1 
explicitly describes 
the use of pictures 
as context clues 
when reading words 
indicates that she 
shares the 
researchers’ 
conception of this 
topic. 










U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education ( NLE ) 
Educational Resources Information Center (ERIC) 



® 




NOTICE 



Reproduction Basis 




This document is covered by a signed "Reproduction Release 
(Blanket)" form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a "Specific Document" Release form. 




This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release form 
(either "Specific Document" or "Blanket"). 




EFF-089 (3/2000) 




